Rainbow

So far, you've learned about three extensions to the Deep Q-Networks (DQN) algorithm:

• Double DQN (DDQN)
• Prioritized experience replay
• Dueling DQN

But these aren't the only extensions to the DQN algorithm! Many more extensions have been proposed, including:

• Learning from multi-step bootstrap targets (as in A3C - you'll learn about this in the next part of the nanodegree)
• Distributional DQN
• Noisy DQN

Each of the six extensions address a different issue with the original DQN algorithm.

Researchers at Google DeepMind recently tested the performance of an agent that incorporated all six of these modifications. The corresponding algorithm was termed Rainbow.

It outperforms each of the individual modifications and achieves state-of-the-art performance on Atari 2600 games!

## In Practice

In mid-2018, OpenAI held a contest, where participants were tasked to create an algorithm that could learn to play the Sonic the Hedgehog game. The participants were tasked to train their RL algorithms on provided game levels; then, the trained agents were ranked according to their performance on previously unseen levels.

Thus, the contest was designed to assess the ability of trained RL agents to generalize to new tasks.

One of the provided baseline algorithms was Rainbow DQN. If you'd like to play with this dataset and run the baseline algorithms, you're encouraged to follow the setup instructions.